EZ: A TOOL FOR AUTOMATIC REDSHIFT MEASUREMENT 
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ABSTRACT 

We present EZ (Easy redshift), a tool we have developed within the VVDS project to help 
in redshift measurement from otpical spectra. EZ has been designed with large spectroscopic 
surveys in mind, and in its development particular care has been given to the reliability of the 
results obtained in an automatic and unsupervised mode. Nevertheless, the possibility of running 
it interactively has been preserved, and a graphical user interface for results inspection has been 
designed. EZ has been successfully used within the VVDS project, as well as the zCosmos one. 
In this paper we describe its architecture and the algorithms used, and evaluate its performances 
both on simulated and real data. EZ is an open source program, freely downloadable from 



http://cosmos.iasf-milano.inaf.it/pandora 



Subject headings: Data Analysis and Techniques, Galaxies, Astronomical Techniques 



1. Introduction 

Thanks to larger telescopes and more pow- 
erful instruments, during the last decade we 
have witnessed an explosion in the size of spec- 
troscopic surveys both in the nearby and in 
the more distant Universe: from the hundreds 
of objects of the surveys carr ied out in the 



( 20091 )') or to the hundred thousand or million 
objects of th e just sta rted programs (VIPERS, 
Guzzol |2009l). Wiggle Z iDrinkwater et al.. (,2009h . 



BOSS, Ischlegel et aP (|2007h 'l and the few mil- 
lions of galax i es of the future (e.g. EUCLID, 



Cimatti et al. 



early nineties (ESP. IVettolani et all (|1997|),CFRS, 
Le Fevre et al.l (|l995l )) to the thousands of galax- 



ies of pre sent days deep sp ectrosco pic programs 



(zCosm os.lLillv et aLl(l2007D . VVDS. lLe Fevre et al. 



spectroscopic 
3), VVDS,Ei 



(|2005[) : iGariUi et al.r(l2008h , DEEP2, ICoil et al. 
(|2004l )) and to the hundred thousands of more 



nearby surveys (e.g. 2dF. IColless et al.l (120011) . 



(I2OO9M 

Such an evolution in survey size requires an analo- 
gous step forward in the way the data are reduced 
and analyzed, making compulsory the use of reli- 
able automatic tools. Among the various tasks to 
be done when carrying out a redshift survey, red- 
shift measurement is one of the most demanding 
in terms of human re sources and skills req uired. 
Historically, the rvsao ( Kurtz fc Minklll998l ) pack- 



SDSS, lAbazajian et al.l (|2009l ). 6dF I Jones et al 



age within IRAF has been the first widely used 



program which has been adopted for redshift mea- 
surement. It makes use of emission hues (emsao 
task) and of the continuum shape {xcsao task), 
on the basis of a correlation of the spectral con- 
tinuum against galaxy or stellar templates, im- 
plementing the algori thm originally proposed by 
iTonrv fc DavisI (|l979l ). Although very powerful, 
it does not foresee to weigh spectral features by 
an error spectrum. When the VIMOS-VLT Deep 
Survey (from here on, VVDS) started, it was im- 
mediately understood that this could be a major 
drawback in using rvsao within that project: VI- 
MOS Low Resolution Red (LR-Red) spectra, like 
those obtained within the VVDS project, are af- 
fected by heavy fringing redwards of '^ 8OOOA. As 
a consequence, spectra show features which are 
purely due to fringing but which can easily be 
mistaken with emission lines by any automatism, 
as well as by humans, unless they are properly 
weighted. Furthermore, if both emission and ab- 
sorption lines are to be used, some scripting is 
required to couple together results from the pure 
cross correlation algorithm and the emission lines 
search one. Last, but not least, the IRAF imple- 
mentation of rvsao did not allow a direct interface 
with the VIP GI package used for VVDS Data re- 
duction ([Scodeggio et aLl boOS). 
For all these reasons, we have decided to develop 
EZ (Easy redshift, pronounced "easy"), a new tool 
which inherits from the rvsao experience and the 
cross correlation technique, but allows to be easily 
plugged within VIPGI and co uld also be used a s a 
VO-compliant tool via SSAP (|Todv et al.lbOOSi) or 
other communication protocols. From the user's 
point of view, the most important requirement 
was to be able to run the program both in a com- 
pletely unsupervised and in fully interactive mode, 
with a user friendly graphical interface to inspect 
the results obtained using different parameters or 
templates. Furthermore, it must be possible to 
use templates which do not necessarily cover the 
full spectral range for any redshift value. Finally, 
it has to be easily usable for different projects, i.e. 
all parameters must be defined either through a 
project dependent parameter file or via command 
line. EZ has been initially based upon the proto- 
types KBred (Scaramella 1999, unpublished) and 
YAZ (Scaramella 2004, VVDS team Internal re- 
port), with which the first 10000 VVDS spectra 
were measured, and which contained the correla- 



tion and fitting algorithm described below. 
In section [2] we illustrate the rationale of EZ. Sec- 
tion [3] gives a short illustration of the software 
architecture and the most important algorithms 
used are illustrated in section S) Section [5] de- 
scribes in full how a redshift can be measured by 
EZ and section |6] illustrates how a reliability flag 
is attached to the best solution proposed. In sec- 
tions [7] and [8] we evaluate the performances on 
simulated and real data respectively. Finally sec- 
tion |9] shows how much time and efforts can be 
spared using a tool like EZ when carrying out a 
large redshift survey. 

2. EZ basic concepts 

There can be several approaches to perform 
redshift measurement: from the most simple ones 
(like emission line finding or cross correlation) to 
more sophisticated ones, like bayesian approach or 
principal component analysis. Within EZ, we have 
chosen to develop a kind of expert system: the core 
of EZ is a decisional tree which tries to mimick the 
decisional pattern followed by astronomers when 
measuring redshifts from spectra. When an as- 
tronomer manually measures a redshift, his brain 
almost unnoticeably performs several functions: 
he looks for the existence of emission lines, and 
if there are some, looks whether they match on a 
single redshift solution; he looks at the shape of 
the continuum, to find out whether the object is 
an early type galaxy, and in that case searches for 
a D4000 break and Ca H and K absorption lines; 
discards parts of the spectrum affected by heavy 
noise and spurious features, like zero orders or sky 
subtraction residuals; possibly, looks at the raw 2- 
dimensional spectrum to decide whether emission 
features are real lines, or sky subtraction residu- 
als. The basic idea we have tried to implement 
in EZ is to allow the user to combine the avail- 
able functions in the most appropriate way for 
the data at hand, thus building new user defined 
functions and methods. At the upmost level, a 
redshift measurement decisional tree can be built, 
which mimicks the decisional path followed by an 
astronomer to get to the measure of the redshift. 
Such decisional path can differ from data set to 
data set, and for this reason several decisional 
trees can coexist in EZ, exactly as several imple- 
mentations {methods) of the same function can 
coexist. 



EZ implements several functions to perform the 
different tasks required: read data files and tem- 
plates, subtract the continuum from a spectrum, 
find emission lines, measure the redshift, measure 
spectral features. Each of these functions, in turn, 
supports different methods, i.e. can use different 
algorithms to carry out that particular task; for 
example, fitting can use a chi-square minimization 
or some more sophisticated algorithm, correlation 
can be performed weighting data by a noise spec- 
trum or without any weight, data can be read 
from fits or ascii files. 

Before coming to a detailed description, it is 
worth mentioning the importance that spectral 
templates have during the redshift measurement 
process. Whenever the solution relies on correla- 
tion or fitting methods (i.e. is not based solely on 
emission lines matching), the reference templates 
used in the process should reproduce as much as 
possible the spectra to be analyzed. It is not al- 
ways straightforward to have at hands templates 
for all possible galaxy types and covering a wave- 
length range as large as to cover from the UV 
to the NIR rest frame. Nor the same template li- 
brary is necessarily the best suited for all projects. 
EZ does not provide templates itself, but allows 
the user to have different template suites and use 
those he thinks are more suitable for that particu- 
lar data set. Furthermore, it is possible to specify 
a minimum range of overlap between the template 
and the spectrum to be used. This feature eases 
a bit the problem of providing templates covering 
an extremely large wavelength range when dealing 
with data sets sampling galaxies from the nearby 
universe up to z > 4. 



3. Software architecture 

EZ is implemented in Python in the form of a 
command-line interpreter. It consists of a main 
Python class which imports other classes dedi- 
cated to redshift estimation, service functional- 
ities, file I/O, line finding, line matching, and 
others. A complete description of the EZ classes 
is provided in the downloadable user's manual. In 
general, only the higher level methods are used 
(e.g. the decisional tree) from within the EZ en- 
vironment, but experienced users can directly use 
the Python shell to import the various classes and 
combine lower level methods in different ways. 



thus exploiting at full the flexibility of the pack- 
age. Even at the higher level, several sessions (i.e. 
instances of the same classes) can coexist, and 
users can directly compare the results obtained 
using different methods or functions. 

Algorithms are implemented in Python, or in C 
for the most CPU intensive tasks: all algorithms 
implemented in C are called through a unique C 
interface, so that all of them can easily be used 
from a Python environment. All the classes are 
built in such a way that they can be imported as 
modules from any other Python code as well as 
from the Python shell. The main class contains 
methods that return the best solution obtained as 
well as a "theoretical" spectrum for a given tem- 
plate and a given redshift normalized to the input 
spectrum. Any other Python code can simply im- 
port the main class and then directly handle the 
proposed solution according to needs. 

While EZ is primarily developed as a command- 
line tool, a gtk based graphical interface is avail- 
able to the user. By design, it is "merely" an in- 
terface to the command-line interpreter: it shows 
the spectrum and the associated noise spectrum, 
allows superposition of the best fitting solution, 
lists the other (less probable) solutions and allows 
to overplot them onto the spectrum. An example 
of the Graphical User Interface is shown in Fig. 
[TJ In this figure we show a low signal to noise 
spectrum with the purpose of demonstrating EZ 
capabilities also on low quality data. 



4. Algorithms 

4.1. Emission line finding 

One way to measure a redshift is to look for 
emission lines on to which to anchor a solution. 
The basic concept of an emission line search algo- 
rithm is to look for sharp peaks in the spectrum, 
as candidate emission lines, and then see whether 
such peaks can be matched with a single redshift 
solution. In presence of fringe patterns which 
highly resemble emission lines, the main problem 
is to discard those peaks which are very likely 
fake features, without loosing true lines. The im- 
plemented procedure first builds a rough peak list 
containing the position of all pixels showing a flux 



above a user defined significance threshold. A 
gaussian is then fitted to each of such positions, 
and only those peaks for which the width of the 
fitting gaussian is within some user defined limits 
are retained: the minimum and maximum width 
of a line depends on the resolution of the spectrum, 
and as such are configurable. This check discards 
peaks due to non or badly cleaned cosmic rays, 
which are too narrow to be a real line. Among 
the remaining line candidates, a further check is 
made whether the real peak flux is within a factor 
of two from the fitted gaussian peak. This allows 
to discard most of the fake peaks due to fringing 
which often have an irregular shape. Depending 
on spectral resolution, partially resolved lines can 
also be poorly represented by a gaussian. For this 
reason such check can be tailored or altogether 
suppressed through user parameters. Finally, the 
significance of the peaks is computed, subtract- 
ing a local continuum and computing the ratio 
between the peak height and the noise weighted 
local continuum: only peaks showing a signifi- 
cance above a pre defined a (usually 4 or 5) are 
retained. As the noise is stronger in fringing af- 
fected regions, spurious peaks with a gaussian 
shape get discarded by this check. The lines thus 
identified are further subdivided into strong and 
normal lines, according to whether the peak value 
is higher than strongcut * a where strongcut is a 
user defined parameter. Strong lines are treated 
differently within the decisional tree (see section 

Finally, the algorithm tries to match all or some 
of such peaks to known combinations of emission 
lines at different redshifts. The lines to be used 
for the matching are defined in a configuration 
file, and can be changed at wish, according to the 
redshift range or the type of object explored by 
the survey. 



4.2. Correlation 

Before applying the correlation both the spec- 
trum and the template are continuum subtracted 
so that what drives the results are the local fea- 
tures (such as weak emission lines, absorption 
lines, spectral breaks). Each available spectral 
template is redshifted to a given redshift and the 
correlation function is computed. During the com- 
putation, each pixel can be optionally weighted 



by its associated noise. In the noise weighted ap- 
proach, the correlation function has the form: 
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where Sj and Uj are the spectrum flux and noise 
at pixel j respectively; s is the spectrum mean 
computed as 

tj{z) is the interpolated flux of the template at 
pixel j, once put at redshift z; t is the template 
mean; A is the wavelength range in use and Q 
is the redshift range to explore. Each template 
does not necessarily cover the full spectral range, 
in other words for different templates and redshifts 
the correlation function can be computed using a 
different number of points. In order to compare 
the results obtained for different templates, the 
value of the correlation function is normalized to 
the number of points used (N) . 
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Once the correlation function is produced for a 
particular template, the highest peak is returned 
as the best correlation solution for that template. 

4.3. Fitting 

The fitting of a spectrum against a template 
is performed using a non-continuum subtracted 
spectrum, so that the results are affected also by 
the overall shape of the underlying continuum. It 
uses a standard least squared metric: each spectral 
template available is redshifted to a given redshift 
and the mean square deviation between spectrum 
and template is computed as 



Xred 
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where fj and ctj are the spectrum flux and noise 
at pixel j respectively; tj{z) is the interpolated 
flux of the template at pixel j, once put at red- 
shift z; A is wavelength range in use and Q is 
the redshift range to explore; N is the number of 



data points used in the computation. Each tem- 
plate does not necessarily cover the full spectral 
range, in other words for different templates and 
redshifts the x^ can be computed using a different 
number of points. In order to compare the results 
obtained for different templates, the x^ is divided 
by the number of data points used The normaliza- 
tion constant A is computed as: 
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The minimum reduced x^ for each template and 
redshift range is returned. 

4.4. The solve method 

When no emission lines have been found in a 
spectrum, or when the lines found do not point to 
one single solution, redshift determination can be 
done using a correlation, or a fitting procedure, 
as described above. The solve function within EZ 
is meant to combine cross correlation, fitting or 
any other elementary method which can lead to 
a redshift solution. As all other functions, it can 
have different methods, i.e. the various elemen- 
tary algorithms can be combined in different ways 
according to wishes. 

The current implementation of the solve method 
foresees the use of cross correlation first, and a 
further fitting step. We have noticed that in the 
VVDS and z Cosmos data sets, the highest peak of 
the correlation function as described above is not 
always the best solution, and this depends mainly 
on the fact that the noise spectrum does not (neg- 
atively) weigh enough the spurious features of the 
fringing. On the other hand, also the simple fit- 
ting does not give satisfactory results in terms of 
picking up the correct redshift. A sequential usage 
of the two methods, instead, has proven to be the 
one giving the best results for our data. First a 
correlation of the spectrum against each template 
is performed: as described above, this step allows 
to properly take into account the local features of 
the spectrum, like spectral breaks, absorption lines 
and weak emission features. At the end of the cor- 
relation, for each template the n redshifts (where 
n is a user defined parameter) corresponding to 
the highest correlation peaks are retained. At this 
point, we are faced with nxm solutions (where m 
is the number of available templates) . To discrim- 
inate among them, we use the fitting procedure: 



as fitting is performed using the spectrum without 
subtracting the continuum, the overall spectrum 
shape plays a role into getting a lower reduced 
X^, and can help to pin point the correct redshift. 
Finally, the solution giving the minimum reduced 
X^ is chosen. 



5. EZ decisional tree 

The core of EZ is the "decisional tree" , which 
tries to mimick the human decisional process ap- 
plied during redshift measurement. This is where 
one defines the actions to be performed, and in 
which sequence they must be carried out in or- 
der to measure a redshift. Each logical block of 
a decisional tree makes use of the lower level al- 
gorithms available. Several decisional trees can 
coexist in EZ: each one acts differently in different 
situations, and this can be useful to better tune 
the measurement process according to the kind of 
data at hand. For example, in a survey where 
stars have been a priori removed, it may be useful 
not to check for M stars, thus reducing the degrees 
of freedom of EZ, and consequently the possibil- 
ity that it takes a false track. As an example, we 
illustrate here the decisional tree we have imple- 
mented for the VVDS and zCosmos surveys. The 
input consists of one or several observed spectra 
with a noise spectrum associated to each of them. 
Following fig [21 the steps performed are as follows 

1. Check if the object is an M type star: the 
characteristical wavy shape of the spectra of 
this kind of stars can be recognized by look- 
ing if around the expected position a very 
large gaussian can be fitted, and if the spec- 
trum is steadily becoming redder and redder 
with increasing wavelength. 

2. If such check is positive, in order to assign 
a best-fitting template to the object a fit is 
made using only M-star templates, and the 
redshift is set to zero. 

3. If the M star check has failed, search for 
emission lines in the spectrum; if lines are 
found go to step HI otherwise go to step [8] 

4. Search for a match between lines. If one or 
more matches with at least 2 lines have been 
found go to step [5l otherwise go to step [S) 



5. Fit each of the possible solutions given by 
the matches above with emission lines tem- 
plates. Choose the solution giving the mini- 
mum reduced x^- 

6. Check if strong lines have been found, (see 
14. ip . If this is the case go to step[71 otherwise 
go to step [8] 

7. Using only the redshifts satisfying matches 
with the one strong line, and only the emis- 
sion line templates, compute the redshift 
using the algorithm defined by the solve 
method 



8. Using all selected templates and the whole 
redshift range indicated by the user, com- 
pute the redshift using the algorithm defined 
by the solve method ()4.4p 

6. Redshift reliability 

Complete automation of the redshift measure- 
ment process can be tricky when spectra are noisy 
(as they always are at the faint limit of a survey) 
or in presence of artifacts such as fringing correc- 
tion residuals, so that it is by no means guaran- 
teed, a priori, that the best solution proposed by 
EZ is also a correct solution. For this reason we 
have added the computation of an integer relia- 
bility flag which summarizes the goodness of the 
solution proposed. Such a flag is computed mim- 
icking the kind of logical reasoning applied by an 
astronomer when trying to evaluate if a redshift is 
reliable or not. The flow is as follows 

• each template is described by a "reliability" 
file. An illustration of such file for an Ellip- 
tical galaxy and for a StarBurst galaxy, as 
used with the VVDS and zCosmos projects, 
is given in Table [TJ in the first column, the 
features which are tested are given, together 
with their wavelength (column 2). Each fea- 
ture has a weight associated (column 5), ac- 
cording to the prominence the feature usu- 
ally has in standard spectra. For example, 
in elliptical galaxies the D4000 break is the 
most prominent feature, thus its weight is 
higher than all other lines. On the other 
hand, if the feature is not found, then the 
redshift becomes suspect, thus a negative 
weight is associated in this case (column 6). 



Some features are commonly found together 
with other features (e.g., the [OIII] doublet, 
or the D4000 which usually comes with Ca 
H and Ca K absorption lines). The third 
column lists such secondary features, if ap- 
plicable. If these features are found together 
with the main line, then the weight of the 
main line is increased to the value listed in 
column 4. Note that what is important is not 
the absolute value of the weight itself, but its 
relative value with respect to the other ex- 
pected fetaures. The reliability files should 
be created according to user's needs and to 
the templates used. 

• the "best fitting" template is redshifted to 
the measured "best redshift" , and a correla- 
tion around each expected spectral feature 
is performed, taking into account the noise 
spectrum. The correlation value must be 
above a user defined threshold for the feature 
being considered as "found". In this step, 
observational constraints are taken into ac- 
count: e.g. if one of the expected lines falls 
outside the observed spectrum (or too close 
to the border) it is ignored. 

• if the "best solution" is a star also the 
"color" is computed, as the difference be- 
tween the mean value in the bluer and redder 
part of the spectrum, and used as a "feature" 
(blue color for earlier star types) . If the star 
type is M, a dedicated algorithm searches 
for the characteristic "wavy" shape and uses 
them as features found (or not found). 

• if a feature has been found, then its cor- 
relation value is weighted according to the 
weight given to that feature in the reliabil- 
ity file for that template (column 4 or 5 in 
Table [1]). Features listed in the reliability 
file, but not found, are negatively weighted 
(column 6 in Table [T]). 

• the weighted correlation values are summed 
up, and normalized by the number of lines 
which have been found, giving a rate 

• finally, the rate is converted into a flag, ac- 
cording to the ratio of found features with 
respect to expected features and to the rate 
itself: the higher the rate and the number 



of features found, the higher the flag. The 
conversion is made in such a way that EZ 
rehabihty flags resemble as much as possi- 
ble the VVDS reliabi lity flags as defined in 
Le Fevre et alj ( 20051 ) in terms of confidence 



level. 

Flags thus computed range from (solution is 
not reliable) to 4 (highly reliable solution). In Ta- 
ble [5] we summarize the confidence level of the dif- 
ferent flags as defined in the VVDS project, and 
the criteria used by EZ to assign each flag. 

7. Performances on simulated data sets 

For any given spectrum, EZ gives in output the 
best redshift and the best fitting template, while 
the output of the reliability process is the flag and 
the number of features found. When evaluating 
performances, we should make a distinction be- 
tween the performances of the redshift measure- 
ment methods, i.e. the capability of finding the 
correct redshift, and the performances of the reli- 
ability method, i.e. the capability of assessing the 
goodness of the solution. 

To perform such evaluations, we have carried out 
two different sets of tests: a first set, using simu- 
lated spectra, and a second set, making use of real 
spectra from the VVDS and the zCosmos survey. 

7.1. Simulated test set 



Using simulated spectra to evaluate perfor- 
mances of an algorithm has the advantage that 
the answer is known beforehand, and thus results 
can be evaluated with no error margin. On the 
other hand, even if simulations are carried out 
as carefully as possible, such set can only give 
an upper limit to the performances, as simulated 
spectra never take into account all possible noise 
sources and defects existing on real data. 
The procedure we used to simulate spectra begins 
with an input catalog of objects: in this case we 
have used the Cosmos Mock Catalog (frorn here 
on CMC), described in I Jouvel et"aiT (|2009l) . In 
brief, CMC is a simulated c atalog built directly 
from the o bserved COSMOS (IScoville et aLllioOTl) 
catalog of ICanak et aD (l2007|) and the CO SMOS 
photometric-redshift catalog (jllbert et al.l [2009) . 
In CMC, a redshift and a spectrophotometric type 
are associated to each galaxy of the COSMOS cat- 



alog, using a model fitting procedure of the pho- 
tometric data. The resulting catalog contains a 
mix of galaxy populations which by construction 
is representative of a real galaxy survey. Emis- 
sion line fluxes are also computed and magnitudes 
in a number of filters are made available. De- 
tails on the simulation of observed spectra will 
be given elsewhere (Franzetti et al., in prepara- 
tion). To summarize the procedure, a rest frame 
spectrum, as can be obtained from galaxy model 
libraries, has been associated to each galaxy type 
as provided in the catalog (elliptical, early spiral, 
late spiral and starburst galaxy). In this step, 
line broadening due to galaxy velocity dispersion 
has been neglected, as at the resolution of the 
VVDS and zCosmos data this it is irrelevant. The 
monodimensional incident spectrum has been ob- 
tained by redshifting the rest frame spectrum and 
normalizing it so as to give the object apparent 
magnitude in the chosen selection band, Iab ■ The 
incident spectrum has then been degraded for the 
VIMOS efficiency curve, as can be obtained from 
the ESO Exposure Time Calculator. In the noise 
calculation, we have taken into account the Pois- 
sonian noise (from both sky and object), the flat 
fielding accuracy and the electronic noise (for a 
detaile d explanation of the different contribution 
see e.g. iNewberrvl (|l99l[ )). The sky spectrum used 
has also been derived from ESO VIMOS exposure 
time calculator. We have compared our simulated 
spectra with those which are obtained from ESO 
exposure time calculator, once the same exposure 
time, galaxy type and apparent magnitude are 
used, and the results are extremely similar both 
in terms of signal to noise as a function of wave- 
length, and in terms of sky subtracted spectrum. 
It is important to note that our simulated data set 
includes the electronic noise, the poissonian noise, 
the flat fielding accuracy but does not include the 
effect of fringing. 

We have simulated ^ 11000 galaxy spectra, in the 
magnitude range 17.5 < Iab < 22.5 using the 
same exposure time as for the VV DS wide sur- 



vey an d the zCosmos bright s urvey (jGarilli et al 



(l2008[ ) and iLiUv et al.l tOOlf t). and the VIMOS 
LR red grism. CMC does not contain stars, but 
we are interested to check EZ performances also on 
different types of stars, as the selection function of 
galaxies for surveys never completely succeeds in 
excluding stars on the basis of photometry. Thus 



we have simulated 11000 stars, with magnitude 
ranging from 17.5 to 22.5, of different spectral 
types. The Pickles stellar library has been used 
for their model spectra. On this simulated data 
set, we have run EZ in totally blind unsupervised 
mode and obtained for each object a redshift and 
a redshift flag. The template set we have used in 
EZ is the one obtained within the VVDS project 
and built from the VVDS data themselves: it 
comprises two templates for early type galaxies, 
one template for Sbc galaxies, one for Scd galaxies 
and 3 for starburst galaxies, with different inten- 
sities of emission lines and line ratios. These tem- 
plates extend from ~ 3500A to about ^ 8OOOA 
rest frame and are particularly suited to search 
for a redshift solution within the redshift range 
to 2.0, a range which well matches the redshift 
range of the simulated data given the magnitude 
cut we have imposed, in spectra covering the ob- 
served range of the VIMOS LR-Red grism. The 
measured redshift has then been compared to the 
real input value, and classified as correct when the 
measured value is within 10~^ of the real value, 
this limit corresponding to the theoretical redshift 
error given by the grism resolution. 



7.2. Global performances on simulated 
spectra 

In Table [3] the results of such comparison are 
summarized: for galaxies and stars separately, as 
well as for the whole sample (which has comparable 
numbers of galaxies and stars) , we give the success 
rate (computed as the ratio between correctly re- 
trieved redshifts and total number of objects), the 
number of correct redshifts, and the total number 
of objects to which EZ has assigned the given flag. 
Results for each object type are splitted per EZ 
redshift flag. 

Table |3] shows that on simulated data, EZ is ex- 
ceptionally good at retrieving the correct redshift, 
the success rate being 97% on the whole sample. 
In spite of the large number of simulated spectra, 
very few objects are classified with a reliability 
flag of 1 or 2, so that the average success rate for 
these flags has not the same statistical significance 
as the other fiags. 

Going deeper in the analysis of results, we note 
that stars are practically always recognized as 
such, even if the fiag associated to their measure- 



ment is extremely low in half the occurrences. 
This is intrinsic to the way we associate flags: for 
types younger than K, stellar spectra are poor in 
spectral features in the wavelength range explored. 
At the resolution we used, only NaD and Ha ab- 
sorption lines are clearly visible. Furthermore, 
NaD falls very near to a strong sky line, and it is 
often not detected due to the higher noise. Thus 
only one out of two features is clearly detectable. 
This explains the frequency of zero flag objects in 
this category. However, only 191 galaxies (< 2%) 
are erroneously mistaken for stars, while among 
the 10762 objects classified as galaxies, only 19 
were actually stars. 



7.3. Dependence on magnitude and red- 
shift 

We have shown that on the simulated test set, 
the global success rate is very high (97%). StiU 
we expect it to show a trend with object magni- 
tude, which can be considered a proxy for signal 
to noise ratio for a fixed exposure time. This is 
shown in figure|31 bottom panel, where we plot the 
success rate obtained as a function of magnitude. 
Black circles show the success rate obtained con- 
sidering all fiags above 1, while red crosses show 
the success rate obtained considering very secure 
redshifts (flags 3 or 4) only. Even using all fiags 
above 1 (secure redshifts), it is evident that the 
fainter objects have an increasingly lower success 
rate, even if it stays always well above 80%. 
The top panel of figure [3] shows the success rate as 
a function of the true redshift of the object. The 
drop in success rate for z > lA for secure redshift 
flags is not very surprising: between z=1.4 and 
z=1.5, both the [OH] line and the D4000 break 
start to fall out of the observed wavelength range, 
and the redshift can be secured only for very few 
galaxies. In other words, we are entering in the 
redshift desert regime, were observations in the red 
visible part of the spectrum are known to be in- 
efficient. Possibly more surprising, at first glance, 
can be the drop in success rate observed above 
z=0.9 when only very secure redshifts are consid- 
ered. This is mainly due to the way we compute 
the reliability of the redshift: at z=0.9, [OIII]a 
and H/3 lines progressively fall out of the observed 
wavelength range, so that the only strong and eas- 
ily detectable line for late objects types remains 



[Oil]. For this reason, raany objects get a flag 9 
(only one strong line). On the other hand, going 
higher in redshift objects become fainter, and their 
continuum gets noisier: therefore also early type 
objects are more difflcult to measure and to get a 
very secure flag is even harder. 

8. Performances on real data sets 



We have tested EZ performanc es on the three 
data sets of the VVDS Deep sur vey (ILe Fevre et al 
20051) . the VVDS Wide su rvev (iGarilli et al.ll2008l)" 
and the zCosmos survey ( Lillv et al.l 120071) . The 
three data sets have different characteristics, and 
are complementary to evaluate EZ performances: 
the VVDS Deep sample is cut to a deeper ap- 
parent magnitude limit {Iab < 24.0), thus its 
redshift distribution extends beyond z—l.b with 
significant numbers. On the contrary, the VVDS 
Wide is cut at a brighter limit [Iab < 22.5), and 
has a stronger star contamination. The zCosmos 
sample, finally, has the same depth as the VVDS 
Wide sample, but it has been observed with a 
higher resolution grism and stars have been a 
priori discarded on the basis of their morpholo- 
gies and spectral energy distributions: the criteria 
used were intentionally quite conservative and a 
small fraction of stars is expected in the sample 
(JLilly et al.ll2007t ). All redshifts for these three 
samples have been manually measured by two dif- 
ferent persons (also using beta versions of EZ in 
interactive way), discrepant measures have been 
reconciled, and a flag has been assigned to each 
redshift, corresponding to a confidence level as de- 
scribed in table [21 column 2, a long and time con- 
suming procedure which has requested big efforts 
on the part of the people involved. A posteriori^ 
we have run EZ in blind and unsupervised mode 
on the data sets, using only objects with a mea- 
sured redshift between and 2.0. By comparing 
the results found by EZ with those published by 
the VVDS and zCosmos consortia, we can eval- 
uate EZ performances on real data. We expect 
such performances to be worse than what ob- 
tained on simulated data sets, as now all possible 
error sources are present, last but not least the 
fringing above ~ 8000A. 

Table S] and Figure S] show the success rate of the 
EZ blind measurement in the three different sam- 
ples. As expected, the global performances are 
not as exceptional as for the simulated data, rang- 



ing from '^ 68% to ~ 76% in the best case of the 
zCosmos data. However, a straight comparison is 
not totally fair: as declared in the surveys them- 
selves and reported in Table [21 measured redshifts 
are never claimed to be 100% correct. The last 
column of Tabic [4] reports the weighted success 
rate: for each EZ flag, the non concordant red- 
shifts have been weighted in number according to 
the humanly given flag, so that discordant objects 
having a human flag 1 count by half, if they have 
human flag 2 or 9 they are weighted 75%, if they 
have human flag 3 and 4 they are weighted 90% 
and 95% respectively. Using such weights, the suc- 
cess rate obviously increases, as the uncertainty of 
the humanly measured redshifts is properly taken 
into account. 

Still, the success rate remains lower than what we 
had for simulated data, especially in the VVDS 
samples. The main reason for this are the fringing 
features, which if not properly weighted are easily 
mistaken for real features by any automatic proce- 
dure. As explained in the previous section, in the 
simulated data set we have not included the effect 
of fringing, and for this reason simulated spec- 
tra are much less noisy redwards of SOOOA and 
EZ is not fooled by spurious emission features. 
The noise spectrum associated to real data should 
help in weighting considerably less such features, 
but when fringing features are particularly strong, 
the noise spectrum, as currently computed, is not 
enough to completely neutralize their effect on 
the emission line finding algorithm. As a test for 
this hypothesis, we have run EZ on these same 
data sets without using the noise spectrum and 
the resulting success rate is considerably lower. In 
the zCosmos sample, thanks to the larger dither- 
ing width the fringing pattern is better removed 
from the data and the associated noise spectrum 
better allows to take into account any residual. 
This is one of the reasons why in the zCosmos 
sample the success rate for the most secure EZ 
flags is the same as for simulated data. A second 
important effect is the presence of the zero order 
in the extracted spectra: if this is not removed, it 
is mistaken for an emission line and the redshift 
is obviously wrong. Again, zCosmos data are less 
affected because the multiplexing of those obser- 
vations is much lower than for the VVDS data. 
Finally, also the observational conditions have an 
impact on the data quality and the redshift mea- 



surement: clouds and bad seeing diminish the S/N 
ratio, a lower exposure time as well as a too high 
airmass decrease the signal to noise, bad centering 
of the object in the slit increases slit losses. All 
these factors contribute to lower the global red- 
shift measurement success rate with respect to the 
idealized case of simulations. 
Looking deeper at the concordant measurements, 
it can be noted that the flag assigned by EZ is 
not always identical to that assigned by the as- 
tronomers. This is summarized in Table [S] where 
we compare EZ flag with the human flag for con- 
cordant measurements. In this table, we use only 
measurements obtained for galaxies, as we have 
already shown that when the object is a star the 
flagging system we have implemented gives low 
reliability values. We have grouped results for 
flags 2 and 9, because experience has taught that 
astronomers' ideas on the two flags differ from 
person to person: when only one emission line 
is clearly visible, some people look at the con- 
tinuum slope and assign a flag 2, others don't. 
Table [5] shows that the similarity of the two flag- 
ging systems is confined to the highest confidence 
bin: the vast majority of the most reliable mea- 
surements by EZ are considered very secure also 
by astronomers. Going to lower EZ flags, the as- 
tronomer's flag is usually higher than what judged 
by the automatic algorithm. This effect is not un- 
expected, since the EZ flagging system has been 
set up to be as reliable as possible, at the ex- 
penses of being too pessimistic in many cases. We 
must also consider that the criteria used by EZ 
and by an astronomer are not exactly the same: 
astronomers only grossly take into account the 
correlation with the continuum slope, while in ab- 
sence of emission lines this is very important in 
an automatic tool. Also the lines to be searched 
must be very significant for EZ to take them into 
account, and the significance is weighted with the 
corresponding noise spectrum, while an eye mea- 
surement is more elastic in judgment, also because 
of the possibility of double-checking ID with 2D 
spectra. Finally, the astronomer's judgment is 
subjective and changes not only from person to 
person, but also with the tiredness of the person 
performing measurements. An automatic evalua- 
tion, on the contrary, although more pessimistic, 
is always based on the same criteria. 
As reported in the papers presenting the surveys 



(iLeFevreet all (l2005l) . iGariUi et al.l (|2008[ ) and 



Lillv et al.l ( 20071 )). a redshift has not been mea- 



sured for all objects. These measurement failures 
are indicated in the catalogs with a conventional 
human flag of zero and no redshift indication, thus 
they are not included in table ID Nevertheless, it 
is interesting to see which flag EZ gives to these 
critical data. In all the three samples we have con- 
sidered, more than 80% of the spectra for which 
astronomers did not measure a redshift get a flag 
or 1 from EZ, a clear indication of a highly un- 
reliable solution. In the remaining few cases, EZ 
gets fooled by fake features, indicating that the 
noise spectrum is not accurate enough. 



9. Application on large extragalactic red- 
shift surveys 

The original purpose for which EZ has been 
developed is to ease and speed up the long and 
painful phase of measuring redshifts in large sur- 
veys, reducing human intervention as far as pos- 
sible while keeping the redshift measurement suc- 
cess rate acceptable for the scientific purpose of 
the project. On the basis of the results shown in 
Table[31 and keeping in mind such purpose, we can 
explore which would be the limitations of adopt- 
ing straightforwardly EZ results in measuring red- 
shifts in a generic redshift survey performed in op- 
timal conditions, as simulated data are, which ex- 
plores the redshift range we have considered here, 
i.e. < z < 2. 

We define as completeness the number of mea- 
sured redshifts over the total number of spectra, 
while purity is the number of correctly recovered 
redshifts with respect to the measured ones. The 
results are given in table [SI accepting blindly EZ 
results for all objects with a measured redshift > 0, 
irrespective of flag, the survey would be > 98% 
complete, with < 6% of wrong redshifts. Accord- 
ing to the degree of completeness which can be sac- 
rificed to purity, it is possible to retain only galax- 
ies with very high reliability flag (3 or 4): as from 
table m in this case the resulting galaxy sample 
would be slightly less than 90% complete, but its 
purity would be extremely high (> 95%). If we ap- 
ply these percentages to next generation surveys, 
where the number of spectra foreseen is of the or- 
der of few hundred thousands, the gain (meant as 
number of spectra for which human measurement 
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can be avoided) in using a reliable automated red- 
shift measurement tool is evident. 
One may argue that real surveys are not always 
carried out in optimal conditions, real data are 
usually more difficult to treat than simulated data 
and as such the percentages given in table |6] are 
only upper limits. In section |8l we have shown 
that EZ is extremely reliable to find the correct 
solution for spectra classified as very secure (the 
success rate being above 90% for EZ flag 3 and 
4) even on real data, affected by different sources 
of noise. Using these encouraging results, we can 
extend the previous considerations to more realis- 
tic sets and estimate what would be the gain, in 
terms of number of spectra which can skip the hu- 
man check, if we would fully trust the redshifts to 
which EZ assigns a flag 3 or 4. This is summarized 
in table [71 where for each of the real samples we 
have analyzed we give the total number of objects 
observed, the number of objects for which EZ mea- 
sured a highly reliable redshift, the purity of this 
highly reliable sample and the gain meant as the 
fraction of measurements which could be avoided. 
Such gain ranges from 40 to more than 50%, and 
it translates into a factor of almost 2 sparing of 
human effort and time. Given the difficult nature 
of the data used for our tests (see the discussion 
on fringing effects in the previous section) such a 
number can be considered as a lower limit to the 
effective gain obtainable by using EZ in a large 
extragalactic redshift survey. 

10. Summary 

We have presented, EZ, an automated tool de- 
voted to redshift measurement. The concept at 
the basis of the tool is the decisional tree, i.e. the 
sequence of operations to be performed to obtain a 
redshift. Such sequence of operations can be cus- 
tomized according to the kind of sample at hand: 
e.g. by discarding any check on stars, or using only 
emission lines without performing the correlations 
on the continua in the case of spectra obtained 
with an instrument operating in slitless mode. 
EZ can be used both interactively, with the help of 
a graphical user interface, or totally blindly in un- 
supervised mode. It is developed in Python, with 
the bulk of computations performed in C to in- 
crease its computational speed. Its Python classes 
can be directly imported in any other Python 
based program, thus making it fully embeddable 



in any application. 

Within EZ we have developed a method to assign 
a reliability flag on the measurement obtained, in 
order to mimic the kind of reasoning done by as- 
tronomers when assessing the goodness of the so- 
lution found. The implemented flagging system, 
though, is rather conservative. 
We have tested EZ on VIMOS-like simulated data, 
and have shown that its performances are excep- 
tionally good, the redshift measurement success 
rate being above 95%. 

We have blindly applied EZ to the VVDS-Deep, 
VVDS-Wide and zCosmos bright samples, and 
have demonstrated how this tool behaves very well 
also on real data. The success rate obtained is 
around 70%, and rises above 90% for redshifts clas- 
sified as very secure by astronomers. 
Finally, we have shown that the adoption of a simi- 
lar tool can save from a minimum of 50% to a max- 
imum of 95% of the redshift measurement load, 
according to the quality of the data and to the de- 
gree of completeness and purity of results one is 
willing to sacrifice in favor of a fast output. 
EZ is now being routinely and blindly run as part 
of the reduction process within the VIPERS sur- 
vey, while a customized version has been set up 
to perform simulatio ns of the E-NIS spectrograph 
(|Cimatti et al.ll2009^ . 
EZ is an open source program, freely downloadable 



from http://cosmos.iasf-milano.inaf.it/pandora 
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Table 1 
Examples of reliability weighting scheme 



Feature wavelength secondary weight if weight if secondary weight if 
name feature all found not found not found 



StarBurst galaxy 



[Oil] 
H/3 

[OIII]a 

Ha 



3727.5 
4861.3 
5006.8 
6562.8 



H7 

[OIII]b 
SII 



10 
25 
30 
25 



10 
10 
10 



-20 
-20 
-20 
-20 



Elliptical galaxy 



D4000 

^band 

H7 

Mgl 

Ca-HFe 
NaD 



4000 
4304.4 
4340.4 
5175.4 
5269.0 
5892.5 



CaH CaK 



50 
10 
10 
10 
10 
10 



20 



-10 








Note. — The weighting scheme presented is the one adopted for the VVDS and 
z Cosmos bright surveys 



Table 2 
Significance of VVDS flags and criteria for EZ flags 



Flag 


VVDS 
confidence 


EZ hnes 


rate 


4 


>95% 


Ndet/Nexp > 0.5 


high 


3 


90% 


Ndet/Nexp > 0.5 


low 


2 


75% 


Ndet/Nexp < 0.5 


high 


9 


75% 


Ndet = 1 


high 


1 


50% 


Ndet/Nexp < 0.5 


low 





0% 


Ndet = 


low 



Note. — Nexp and Ndet indicate the num- 
ber of expected and detected spectral features 
respectively. See section [6] for a detailed expla- 
nation. 
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Table 3 
ez performances on simulated data 



EZ flag 


SR 


Ncorrect 


Ntotal 




Galaxies 


any 


92% 


10092 


10934 


3-4 


95% 


9307 


9709 


2 


35% 


14 


40 


9 


98% 


708 


717 


1 


29% 


43 


144 





6% 


20 


324 




St 


ars 




any 


99% 


8103 


8122 


3-4 


99% 


4320 


4321 


2 


99% 


129 


130 


1 


100% 


3 


3 





99% 


3651 


3668 




Galaxies 


and stars 




any 


97% 


18196 


21955 


3-4 


97% 


13627 


14030 


2 


91% 


143 


170 


9 


98% 


708 


717 


1 


42% 


46 


147 





93% 


3671 


3992 



Note. — The success rate (SR) is de- 
fined as the percentage of simulated 
spectra (Ntotal) that are assigned a cor- 
rect redshift (Ncorrect). These are tab- 
ulated as a function of the EZ fiag de- 
scribed in the text. 
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Table 4 




EZ PERFORMANCES 


ON REAL 


DATA 


EZ flag 


SR 


Ncorrect 


Ntotal 


SR weighted 




VVDS Deep 


any 


68% 


5833 


8514 


76% 


3-4 


91% 


3845 


4191 


94% 


2 


51% 


220 


427 


61% 


9 


75% 


730 


962 


82% 


1 


48% 


372 


762 


58% 





30% 


666 


2172 


40% 


VVDS Wide 


any 


70% 


12242 


17436 


78% 


3-4 


89% 


9091 


10126 


93% 


2 


62% 


810 


1288 


72% 


9 


51% 


263 


509 


59% 


1 


50% 


698 


1381 


56% 





33% 


1381 


4132 


43% 


zCosmos Bright 


any 


76% 


6403 


8404 


80% 


3-4 


97% 


3735 


3822 


98% 


2 


87% 


445 


510 


90% 


9 


57% 


194 


336 


62% 


1 


53% 


677 


1260 


59% 





54% 


1352 


2476 


62% 
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Table 5 
ez and humanly assigned flags comparison 





human fla 


Lg 


EZ flag 


3-4 


2-9 


1 


VVDS Deep 


3-4 


84% 


15% 


1% 


2-9 


42% 


54% 


4% 


1 


55% 


41% 


4% 





39% 


52% 


9% 


VVDS Wide 


3-4 


72% 


24% 


3% 


2-9 


26% 


61% 


12% 


1 


39% 


52% 


9% 





24% 


56% 


20% 


- 


sCosmos 


Bright 




3-4 


89% 


9% 


2% 


2-9 


58% 


36% 


6% 


1 


74% 


20% 


5% 





40% 


44% 


15% 



Table 6 
Completeness and purity of the simulated galaxy sample 

flag Ninput Nmeasured Ncorrect Completeness Purity 



any 


10934 


10762 


10092 


98.4% 


93.8% 


3-4 


10934 


9709 


9307 


88.8% 


95.8% 


2;3;4;9 


10934 


10466 


10029 


95.7% 


91.7% 



Note. — Completeness is deflned as the number of measured red- 
shifts over the total number of input spectra, purity is defined the 
fraction of correctly recovered redshifts with respect to the measured 
ones 
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Purity 


Table 7 
of the real data sample 




sample 


Ntotal EZ flag 3-4 


purity 


gain 


VVDS-Deep 

VVDS Wide 
zCosmos Bright 


9742 3845 
18984 10126 
8404 3822 


91% 
89% 
97% 


40% 
53% 
45% 



Note. — Gain is defined as the percentage of spectra for 
which human measurement can be spared 



17 




START 




Check 
Mstar 



Use only M star templates 

Z=0 

Fit 



Find lines 
N=N. of lines 




Solve 

Use all templates 

Use all redshifts 



Match lines 

M=N. of matching lines 




Use only emission templates 
Use only matching redshifts 
Fit 



Match strong lines only 
S=N. of matching strong lines 




Solve 

Use only emission templates 

Use only matching redshifts 




Fig. 2. — Example of EZ decisional tree 




Fig. 3. — EZ success rate as a function of magni- 
tude (lower panel) and of redshift (upper panel), 
considering flags 2 and above (black open circles) 
and only flags 3 and 4 (red crosses). Horizontal er- 
ror bars indicate the bin width, vertical error bars 
are poissonian errors 



Fig. 4. — EZ success rate as a function of reliabil- 
ity flag. Black dots for VVDS Deep sample, red 
crosses for VVDS Wide sample , green squares for 
zCosmos bright sample 
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Fig. 1. — EZ Graphical User Interface. In this figure we show a low signal to noise spectrum with the purpose 
of demonstrating EZ capabilities also on low quality data. Top panel: observed spectrum with best fitting 
template superimposed (red line) . Most important emission or absorption lines are shown as vertical dotted 
red lines, emission lines found by EZ are marked with vertical blue dashed-dotted line. Middle panel: the 
noise spectrum. Bottom panel: the best reduced x^ found for each template as a function of redshift. Left 
panel: top, pull down menu with the list of currently loaded spectra; middle, the list of available templates, 
the one currently shown is highlighted; bottom, the redshift of the currently displayed solution (default to 
the best solution found) and the corresponding reduced x^ 
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